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We study entropy of chaotic oscillation of electrical currents in the Chua's circuit con- 
trolled by triggering a pulse that brings the orbit that goes onto an unstable branch back to 
a stable branch. A numerical simulation of the voltage of the two capacitors and the current 
that flows on an inductor of the Chua's circuit reveals various oscillation patterns as the con- 
ductance that is connected between the two capacitors and directly connected to an inductor 
is varied. At small conductance, the Lissajous graph of the voltage of the two capacitors 
shows a spiral, while at high conductance a double scroll pattern appears. The entropy of 
the current that flows on the inductor is alocal minimum in the spiral state which is in the 
steady state, while it is maximum in the stable double scroll state. The stable double scroll 
samples are analyzed by using the Hidden Markov Model (HMM) and the eigenvectors of the 
transition matrix of long time series are found to be strictly positive but those of unstable 
short time series have negative components. We thus confirm maximum entropy production 
in the double scroll of the longest time series around the right fixed point, while the local 
minimum entropy production occurs in the spiral around the left fixed point. 

§1. Introduction 

In analyzing nature of complex systems it is important to know whether the 
entropy of the system is maximum or minimum. We studied in^' the chaotic oscilla- 
tion of the Chua's circuit, which consists of two capacitors, inductor, two diodes, 
a variable resistance and an opeamp. The opeamp produces piecewise-linear but 
bending resistance, and thus the system is non-linear and when the variable resis- 
tance that is located between the two capacitors is decreased, the Lissajous graph of 
the voltage of the capacitor 1 and that of the capacitor 2 changes from a spiral to a 
double scroll. The oscillation becomes chaotic when the conductance of the variable 
resistance increases, but we showed irP that the chaotic oscillation of currents can 
be controlled by triggering a pulse when the voltage of a capacitor in the Chua's 
circuit passes a certain voltage from the larger absolute value (right) to the lower 
absolute value (left). The pulse trigger violates the left-right symmetry, and the 
spiral occurs around left fixed point. The orbit returned from around right fixed 
point and which tend to move on the unstable branch is kicked by the pulse to one 
on a stable branch, and we could generate various oscillation patterns of the time 
series. 

We performed a simulation of the Chua's circuit and found that when conduc- 
tance of the variable resister is larger than a critical value that changes the orbit 
from a spiral to a double scroll, but relatively smaller than another critical value 
from which a chaotic oscillation begins, there appears an orbit whose length of the 
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time series is long. But when the conductance becomes higher, transition to unstable 
orbit occurs and the period that the orbit is on the stable manifold becomes short. 

In this paper, we analyze the entropy of the oscillation pattern of Chua's circuit, 
since whether the maximal entropy state and/or minimum entropy state can be 
detected is an important problem in the study general features of complex systems. 

Feynman said in the Lectures on Physics,® "If currents are made to go through 
a piece of material obeying Ohm's low, the currents distribute themselves inside the 
piece so that the rate at which energy is generated is as little as possible. Also we 
can say (if things are kept isothermal) that the rate at which energy is generated is 
a minimum." 

In the statistical physics, internal energy E, entropy S and the volume V satisfy 
the relation 



and at the constant pressure, dS = ^dE. 

When an electrical circuit is attached to a battery and the heat Q and the work 

W are supplied to a system, the sum of the two makes the internal energy U and 

dQ 

the heat divided by the absolute temperature T, = dS is called the entropy 

change. When local equilibrium exists in the system, the entropy change consists of 
that due to exchange of energy and/or matter with external system dSe, and that 
due to irreversible processes in the system dSf. dS = dSe + dSi. 

In the textbook on statistical physics of Landau and Lifshit^ it is claimed that 
in the equilibrium states, there are stable and metastable states and that the sta- 
ble state produces the local maximum of the entropy. In non-equilibrium systems, 
Prigogine^' showed that in the linearly interacting systems, the rate at which a sta- 
ble, steady state, non-equilibrium system produces entropy internally is a minimum 
(minimum entropy production rule).'^ One could imagine that, when the temper- 
ature of subsystems is the same, minimum energy generation corresponds to the 
minimum entropy generation. 

When there is a sudden temperature variation, the minimum entropy produc- 
tion rule even sufficiently close to the equilibrium is inadequate.'^ The validity of 
entropy production principles for linear electrical circuits was discussed by Bruers 
et al.'^ They started from the Langevin equations obtained by the Kirchhoff 's laws 
with a Johnson-Nyquist noise at each dissipative element on linear electrical circuits 
and pointed out that the time-reversal odd physical component such as the current 
discussed inP*' does not follow the minimum entropy production rule. 

Dewa^ emphasized the constraint such as Kirchhoff 's rule in the derivation of 
the entropy production rules. He derived the maximum entropy production by using 
the maximum path information entropy production method, and Bruer^^^I showed 
the relation between the equilibrium maximum entropy production in information 
system and non-equilibrium maximum entropy production in statistical systems. 
He showed how to derive minimum and maximum entropy production in statistical 
systems. 

In the system of Chua's circuit, we consider the energy E as VI = RI^ and 
is measured by using the Fourier transform and the Parcseval's formula. Due to 
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Joule heating from the resistance and the pulse, the minimum entropy production 
rule could be violated, but from the Feynman's conjecture and Prigogine's theorem, 
the entropy of the stable spiral steady current is expected to be a local minimum. 
In the case of capacitors, the entropy creation rate i^ 

diS__Vcl_ VcdQ 
"dt ~ ^ ~ ~T~dt 

- 1 d CVc^ . 

~ ~Tdt^ 2 

In the case of inductor, 

diS 1 d ,LI^, LIdl VLI , , 

Since we trigger a pulse when the orbit tends to escape from the stable manifolds, 
we consider ensembles of different time length. Entropy production and escape rates 
was discussed \v^^ They start from stationary ergodic Markov process with the 
state > at time t. They fixed r > and the average of X^-^X^t, - ■ ■ ^X^ 

is defined as the time average. The states of the Markov process are defined as 
intervals [^/e, i?/e] € R, where e defines a scale such that x = eX G [0, 1] and the 
probability that the system has a value eX = x at time t is defined as p{x,t). In 
the stationary state p{x) is assumed to be proportional to e~^^^^^'^, where S{x) is 
the entropy function. They showed a relation between the entropy function and the 
Hamilton's principal function. 

Orbits of Markov process on stable and unstable manifolds are studied in the 
system of hyperbolic differential equation. When an orbit is close to the fixed 
point defined by the process, Perron-Frobenius theorerrfHl' asserts that the transition 
matrix has a positive eigenvalue and its eigenvectors have all positive components. 
When there are intersections of stable and unstable manifolds, the theorem is mod- 
ified by Ruell (Ruelle's Perron-Frobenius theorem) such that theorem applies when 
the sequence is long enough.ll^ Since tangencies between the stable and unstable 
manifolds occur frequently,!!^ the theorem could depend on the nature of the fixed 
point. 

In speech recognitions and pattern recognitions of time series's data, the Hidden 
Markov Model (HMM) which is an application of the Bayes statistical theory is well 
known. In this method, one tries to determine hidden parameters from observed 
parameters. The extracted model parameters are used to perform further analysis 
and parameters that define the time series of the Markov Process in the equilibrium 
are derived by iteration.lI3''I^ Application of the HMM to double scroll time series 
of Chua's circuit was tried in 90'&'^^' but not successfully, since they could not predict 
to which fixed point the orbit will approach in the future. Since we trigger a pulse 
when a orbit around right fixed point passes to unstable branch, the unstable orbits 
around right fixed point are suppressed, and one can obtain parameters of stable 
orbits around right fixed point. 

We apply the HMM in the analysis of the time series of the voltage of the 
capacitor 1 {vqi), that of the capacitor 2 {vc2) and the current on the inductor 
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(il)- We record the data sets of {vci,vc2,'i'L}, and consider the Poincare surface 
specified hy ii = ±GF, where G is the conductance and F is defined from the 
fixed point of the double scroll orbits {vci,vc2) = {^F,0). We define the distance 
between the point at ti on the orbit and the fixed point on the Poincare surface 



x{ti) = Y (F — l^ci 1)^ + V(^2 where U is the time when the orbit crosses the Poincare 
surface. We also measure the binary (0,1) sequence bi,b2, ■ ■ ■ whether the sign of 
vci is positive or negative between ti and tj+i. 

Since the HMM measures the forward time series and backward time series 
towards fixed points separately, we study the dependence of the Perron-Frobenius 
theorem on the fixed points of the double scroll and correlation to the length of the 
time series between the pulses triggered as the orbit passes a plane in the phase 
space. 

The contents of this paper are as follows. In sect. 2, we present the differential 
equation used in analyzing the Chua's circuit and in sect. 3, the result of entropy 
analysis. In sect. 4, the HMM applied to the double scroll system is explained and 
in sect. 5, the result of HMM is summarized. Discussion and conclusion are given in 
sect. 6. 



Chua's circuit consists of an autonomous circuit which contains three-segment 
piecewise- linear resistor, two capacitors, one inductor and a variable resistor. 
The equation of the circuit is described by 



where vci and vc2 are the voltages of the two capacitors (in V), il is the current 
that flows in the inductor (in A), Ci and C2 are capacitance (in F), L is inductance 
(in H) and G is the conductance of the variable resistor (in f2~^). The three-segment 
piecewise-linear resistor which constitutes the non-linear element is characterized by, 

g{vci,rnO,ml) = (mi - mo){\vci + Bp\ - \vci - Bp\)/2 + niovci, 

where Bp is chosen to be IV, mo is the slope (mA/V) outside \vci\ > Bp and mi is 
the slope inside —Bp < vci < Bp. The function ^(uci, mo, mi) can be regarded as 
an active resistor. 

As the conductance between the two capacitors is increased, the spiral changes to 
double scroll. Details of the Chua's circuit with impulse generator and its bifurcation 
analysis are shown in^' and the network is shown in Fig|TJ 

Experimental data of the transition from spiral to double scroll as a function of 
the variable resistance (R8 in the Fig{T]) are shown in Appendix Al. 

Simulation data of entropy production from the capacitors and the inductor are 
shown in Appendix A2. As examples of the orbit, we take G = 0.7429 which is close 
to the transition point of spiral to double scroll, and G = 0.7052 which is in the 




§2. The differential equation of the Chua's circuit 
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Fig. 1. Chua's circuit with a pulse generator used in the experiment of! 



double scroll area. The orbits in {vci,vc2,iL} space are shown in Fig 12] and in FigJS] 
respectively for G = 0.7052 and for G = 0.7429. 





Fig. 2. An example of the orbit in {uci , 1^02, 2l} Fig. 3. An example of the orbit in {wci , ^02 , it } 
space. (G = 0.7052) space. (G = 0.7429) 
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§3. The entropy analysis of the electrical current 

For an oscillation of a period T, the expansion of a time series's data f{t) is 
given as 



oo 



f{t) = + COS 2T:kt + bk sin lirkt) 

k=l 



2 /■■^ 

ak = — I f{t) cos 2'Kktdt 

1 Jo 

2 /""^ 

6fc = — / f{t) sin 2Tr ktdt 
I Jo 



and by applying the Parseval's formula, we obtain 

rT „2 



I fm = T^ + W{al + bl) (3-1) 

7 1 



k=l 

The integral over a period T becomes / —I{t)'^dt, / — V{t)'^dt, / — I{t)^dt. 

Jo 2 Jo 2 Jo 2 



We plot the zero-mode subtracted average 

5c = :^j[^nt)^-| = ^f;(4 + 6l),etc. 

We measure the average entropy of the electrical current between the periods 
when the pulses are triggered. The zero- mode subtracted entropy as a function of 
the length of the period of G = 0.7052 is shown in Fig. HI and that of G = 0.7429 is 
shown in FigjSl We define the orbits around the left fixed point as W^{L) and the 
orbits around the right fixed point as W^{R). The orbits start from around the left 
fixed point. 

The zero-mode contribution to Sl is larger in the case of G = 0.7429 than in the 
case of G = 0.7052. When the length of the double scroll is the longest, its entropy 
becomes maximum, which suggests Ruell's Perron Frobenius theorem. In order to 
extract parameters that characterize the oscillation patterns, we assemble samples 
whose period T is longer than 1000. An example of the ordinary orbit of G = 0.7052 
is shown in Figl8]and that of G = 0.7429 is FigEl . 

In addition to the orbit whose entropy increases monotonically, there is a branch 
whose entropy reaches a maximum and then decreases as shown in Figl6]and in FigjTl 
The decrease of the entropy means the orbit approaches a fixed point whose entropy 
is local minimum. These two branches are separable in the case of G = 0.7052 
as Fig2] and Figi6j When G = 0.7429, two orbits overlap as Figj5] and FIT] show. 
The typical double scroll orbit of G = 0.7052 in Figg] and that of G = 0.7492 in 
Figj9] indicate that long circulation around right fixed point appears freqently in 
G = 0.7052 but not in G = 0.7492. FigJlOland Figllllare examples of long and short 
orbits in G = 0.7492. 
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Fig. 4. The zero-mode subtracted entropy of 
the G = 0.7052 system as a function of the 
length of the time series. The diamond is 
the entropy of the current Sl, the disk is 
that of the capacitor 1 Sci and square is 
that of the capacitor 2 Sc2- The orbits 
on W"(i?) that rotates around the right 
fixed point shows a local maximum of the 
entropy. 
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Fig. 5. The zero-mode subtracted entropy of 
the G = 0.7429 system as a function of the 
length of the time series. The meanings of 
the symbols are the same as FigU) 
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Fig. 6. The zero-mode subtracted entropy of 
an orbit on W%L) of the G = 0.7052 sys- 
tem as a function of the length of the time 
series. 
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Fig. 7. The zero-mode subtracted entropy of 
an orbit on W"(L) of the G = 0.7429 sys- 
tem as a function of the length of the time 
series. 



§4. The HMM analysis of the electrical current 

In general, a sequence of events is a Markov process when the time series is 
defined by the state of one step before. When one restricts chaotic oscillation of the 
voltage of capacitors to the stable double scroll, and by taking into account the pulse 
that kicks the current on the unstable orbit to a stable orbit of the double scroll, 
one could make the system satisfy the detailed balance, and the time series of the 
stable double scroll can be regarded as a Markov chain. In this paper, we restrict 
the analysis of samples whose period T is longer than 1000. All the orbits of this 
sample are assumed to be absorbed in a limit orbit. 

We identify the position of the orbit at time t following Hayes, Grebogi and 
Ott,'^ in which the distance of the position on the orbit from the fixed point is 
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Fig. 8. The orbit on Vy"(_R) and W'(L) of the Fig. 9. The orbit on Vy"(i?) and of the 

G = 0.7052 system. G = 0.7429 system. 








Fig. 10. The long orbit on W°{L) of the G = Fig. 11. The short orbit on W^L) of the G = 
0.7429 system(4c). 0.7429 system(lc). 

calculated as 

x{t,) = ^(F-\vci{tjW + vc2{t,r (4-1) 

and specify around which point it circulates by a coding function r(x) which is 
defined as follows. 

We define the binary time series 6i , 62 , &3 1 ■ ■ ■ assigned at each time when the 
orbit crosses the Poincare surface of i/, = zizGF, according to the voltage of the 
capacitor vci is positive(l) or negative(O). From the data of bn, we define 

00 

r = ^6„2-". (4-2) 

n=l 

The symbols that occur in earlier times are given greater weight. 
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Using the list R = r(l), r(2), • • • ,r{T) and the list X = xi,X2,--- ,x-j-, we 
denote by Wi the pair of list (r(t), and the probability that the system is in this 
state as 

P(n\S) = QM!^.). (4.3) 

We call the state at t = ti when the orbit passes the Poincare surface and the 
measurement of the oscillation states as i and the state at t = t2 when the orbit 

passes the Poincare surface at the next time as j. The transition matrices aij for 
arbitrary j can be derived from the sample average when samples of the similar 
length T are assembled. 

We define a model M that the system makes a transition from a state 1 to a 
state 2 and output the data xi, then makes a transition from 2 to a state 3 as giving 
an output X2, and then output X3, and the probability 

P{X, S\M) = ai2b2i,xi)a22b2{x2)a23b3{x3) ■■■ ■ (4-4) 

We try to maximize the probability 

r 

P{X\M) = a^(o)r(l) n ^(t)(^t)«r(t)r(t+l)- (4-5) 
R t=l 

Wc define the distribution of the observation symbol bj(k) as the probability 
that the system in the state j gives the output y^, which is calculated by integrating 
the distribution given by a list vector bj{xt). 

4.1. Forward probability 

When the number of states of the model M is N, the forward probability 

at{j) = Pixu--- ,xt,r{t)=j\M) (4-6) 
can be evaluated from the recursion 

N 

0!t+i{j) = [J2^t{i)(^ij]bj{xt+i) (4-7) 

i=l 

The initial condition is such that the probability that the system is in the state 
i is TTf ai{i) = 7ribi{xi). In the calculation at+i, bj{xt) is a sum of the components 
of a list vector given from the distribution of x at time t. 

The Qij and bj{k) are updated in the backward probability calculation. 

4.2. Backward probability 

The backward probability Pt{j) is defined as 

= P(yt+i, ■ ■ ■ , yrW) = j, M) (4-8) 

The initial condition ji^ii) = bj-{i) is defined from the sample average at t = T- 
(3t{i) is calculated by the recursion 

N 

(5t{i) = aijbj{xt+i)(3t+i{j), (4-9) 
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where bj{xtj^i) is a list vector, and (3t-i{i) is the sum of the components of the hst 
vector. 

When the model M is specified by A = {R,X,tt), the probability the data x is 
output is 

N N 

at{i)aijbj{xt+i)(3t+iij) (4-10) 

i=l i=l 

We define 

= Pr{x\R,X,n) ' ^^'^^^ 

and7t(z) = Ylf=i^t{hj)- The probabihty Pr(x|i?, X, vr) is a sum of the probabiHties 
such that the components of the hst vector bj{xt+i) are summed up. 
The transition matrix Uij is updated as 



The distribution bj{k) is updated as 



(4-12) 



^.(^) = ^ife^^ (4.13) 
§5. The results of the HMM analysis 



Among samples with the same conductance but different initial conditions, we 
pick up about 10 samples of the data of G = 0.7052, whose length of the period T 
is larger than 1000. 

The states of the time series are assigned by r which runs from 0(state 1), 
l/32(state 2), 3/32(state 3), 7/32(state 4), 15/32(state 5), l/2(state 6), 31/32(state 
7) and the final state (state 8). Since the initial state has r = ^, the first state is 
assigned as the 6th state, and since the next state has r = 0, the second state is 
assigned as the 1st state. 

The state in the 6th state is transferred to the 1st state with the probability of 
100%. Then in the next step it stays in the 1st state with the probability of 96% 
and goes to the 2nd state with the probability of 4%. This kind of information is 
contained in the matrix Ojj. 

The transition matrix Uij obtained from the long T samples is shown in Table 

m 

When the samples are restricted to short T samples, we obtained the Uij matrix 
is shown in Table HIl 

We define the probability bt{i) such that when the orbit passes a point xt on 
the Poincare surfaces at the time 't', the state is in the 'i'. The sequence dependent 
state probability bt{i) of G = 0.7052, long T samples are given in Table Hill The 
corresponding data of short T samples are given in Table IIVI 
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Table II. The aij matrix of the G — 0.7052 short T samples. 



In the HMM, the values of a^j and the ht{i) in the equihbrium are obtained by 
iteration. The Perron-Frobenius theorem says that a dynamical system with a finite 
number of states can be guaranteed to converge to an equihbrium distribution p* , if 
the computer algorithm is Markovian, is ergodic and satisfies detailed balance.'^ 

Before iteration, the a^j of the long T samples has three non-zero eigenvalues 
which are 1, 0.96 and 0.4. After one iteration ai2 = 0.044 and ayg = 0.6 changes 
to and nonzero eigenvalues become 1,1 and 0.18. Although an eigenvector of 
before iteration has negative components, after an iteration, all the eigenvectors have 
positive components. It is a general property that a Markov chain should satisfy due 
to the Perron-Frobenius theorem.tHl' 

In the case of short T samples, non-zero eigenvalues before iteration are 1, 
0.850 and 0.217. Main difference from the long T samples is that the sign of two 
components of an eigenvector becomes negative, which suggests that the system is 
unstable. 

The qualitative difference of G = 0.7052 and G = 0.7492 can be understood in 
the Markov partition. A rectangle R in the phase space of the orbits is covered by 
proper rectangles that satisfy for x in a rectangle R and the diffeomorphic map /, 

• (a) int(i?j) n \at{Rj) = (j) 

• (b) fwlx^Ri) D VF"(/x,i?j)and 

fW'{x,R^) C W'ifx, Rj) whenx G mt{Ri)Jx € mt{Rj). 
In the covering of the rectangle {Ti,T2, ■ ■ ■ ,Tr}, Tj CiTi^ (p, one could separate 

= {x£ Tj : Wix, Tj) nn^cP, W%x, Tj) n = 0} and 
rf,^ = {xe Tj : W^ix, Tj) nTk = <P, W'ix, Tj) n Tfc / 0} 
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0. 


0. 


0. 


0. 


0. 


13 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


14 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


15 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


16 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


17 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


18 


0.8 


0.2 


0. 


0. 


0. 


0. 


0. 


0. 


19 


0.8 


0. 


0.2 


0. 


0. 


0. 


0. 


0. 


20 


0.7 


0.1 


0. 


0.2 


0. 


0. 


0. 


0. 


21 


0.7 


0. 


0.1 


0. 


0.2 


0. 


0. 


0. 


22 


0.4 


0.3 


0. 


0.1 


0. 


0. 


0.2 


0. 


23 


0.4 


0. 


0.3 


0. 


0.1 


0. 


0.2 


0. 


24 


0.3 


0.1 


0. 


0.3 


0. 


0. 


0.1 


0.2 


25 


0.3 


0. 


0. 


0. 


0.3 


0. 


0. 


0.4 



Table III. The sequence dependent state probability btii) of G = 0.7052 long samples. 

in our discretized time series's data of G = 0.7052. But we had the impression that 
the wild hyperbohc set^lSli i_g_ tangency of unstable and stable manifolds appear 
quite often in G = 0.7492. 

§6. Discussion and Conclusion 

We studied the entropy of the electrical current and the parameters of the HMM 
of the Chua's circuit. We observed that the entropy of the steady spiral is a local 
minimum and that of the stable double scroll is a local maximum and the sample 
that has longest period T has the maximum entropy. There appears samples, which 
change their double scroll orbits to spiral-like orbits which make also a local maxi- 
mum of the entropy. Restricting samples to those of the stable double scroll which 
have the period T longer than 1000 in the case of G = 0.7052, we could make the 
time series satisfy the Markov chain condition and obtain the transition matrix aij 
whose largest eigen value is 1 and the components of the eigenvector are positive. 

The Perron-Probenius theoreirPSJ-CSl' says that for a square nonnegative matrix 
some power of which is positive, there is a simple root A of the characteristic polyno- 
mial which is strictly greater than the modulus of any other roots, and A has strictly 
positive eigenvectors. In the proof of this theorem, Brouwer's fixed point theorem is 
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t\i 


1 


2 


3 


4 


5 


6 


7 


8 


1 


0. 


0. 


0. 


0. 


0. 


1. 


0. 


0. 


2 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


3 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


4 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


5 


1. 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


6 


0.142857 


0.857143 


0. 


0. 


0. 


0. 


0. 


0. 


7 


0.142857 


0. 


0.857143 


0. 


0. 


0. 


0. 


0. 


8 


0.0357143 


0.107143 


0. 


0.857143 


0. 


0. 


0. 


0. 


9 


0.0357143 


0. 


0.107143 


0. 


0.857143 


0. 


0. 


0. 


10 


0. 


0.0357143 


0. 


0.107143 


0. 


0. 


0.857143 


0. 


11 


0. 


0. 


0. 


0. 


0.107143 


0. 


0.0714286 


0.821429 


12 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


1. 


13 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


1. 


14 


0. 


0. 


0. 


0. 


0. 


0. 


0. 


1. 



Table IV. The sequence dependent state probability bt{i) of G — 0.7052 short samples. 



used. 

In the short T samples of the time series of G = 0.7052, flows to the equilibrium 
stable spiral state and non-equiliblium stable double scroll occurs, and we find that 
the eigenvectors of aij of short T sample is not strictly positive. The topology of the 
underlying dynamical system invalidates the application of the Brower's fixed point 
theorem in this case and introduction of branched manifold and Markov decomposi- 
tion of the manifolcP^I'lEJ jg necessary. We think the long T samples of G = 0.7052 
are in the near-equilibrium linear response regim^^^ and the orbit in the asymptotic 
state is in W^{R) while the asymptototic states of samples of G = 0.7429 are in 
W'^{L) but the tangencial transition to the manifold with W'^{L) is frequent. 

Although we studied a specific dynamical system of electrical circuits, it is the 
first successful application of HMM to the chaotic double scroll system. There are 
interesting problems generic to non-equilibrium dynamical systems. We confirm 
maximum entropy production in the double scroll orbits in the near-equilibrium 
linear response regime, while the entropy production in the spiral is found to be a 
local minimum. 

Properties of the fixed point is important also in a non-linear system like QCD. 
Simulation analyses suggest that there is an ultraviolet fixed point and an infrared 
fixed point. Gauge invariance of the ultraviolet fixed point is established, but the 
structure of the infrared fixed point is complicated due to the Gribov copies and 
relaively large color anti-symmetric part as compared to the color symmetric part 
in the ghost propagator.l^ Among many orbits tending to a local minimum, one 
tries to find a unique gauge. If the analogies of the ultraviolet fixed point and the 
right fixed point of the double scroll, and the infrared fixed points and the left fixed 
points of the double sroll work, further analyses of the Chua's circuit will allow to 
find a way to pick up an orbit that flows to a unique gauge when there are many 
local minima, or will assert its difficulty. 
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Appendix 

In this appendix, we show an example of the transition from a spiral to dou- 
ble scroll observed experimentally on the Chua's circuitf^ and the entropy of the 
capacitors 1, 2 and the inductor measured by the simulation.'^ 

Al. The transition from a spiral to double scroll 

When conductance is small, the transition from an orbit around one fixed point 
to an orbit around another fixed point does not occur and the spairal pattern appears 
as shown in Table |Vl The number of cycle increases as the conductance increases 
and transition to double scroll occur. The number of circles around left fixed point 
and around the right fixed point becomes unequal at some conductance. The symbol 
L3-R4 means 3 circulations around the left fixed point and 4 circulation around right 
fixed point. The experimental data suggest that a long orbit that circulates around 
the right fixed point occurs when the conductance is small. The simulation data of 
G = 0.7052 corresponds to this case. 



type 


resistance {R[kn]) 


No. of periods 


conductance (1/-R) 


spiral 


1.614 


1 


0.620 


spiral 


1.543 


2 


0.648 


spiral 


1.536 


4 


0.651 


spiral 


1.525 


3 


0.656 


double scroll 


1.435 


6 


0.697 


double scroll 


1.418 


5 


0.705 


double scroll 


1.396 


4 


0.716 


double scroll 


1.390 


L 3,R 4 


0.719 


double scroll 


1.368 


3 


0.731 


double scroll 


1.355 


L 3,R 2 


0.738 


double scroll 


1.346 


2 


0.743 


double scroll 


1.331 


2 


0.751 


double scroll 


1.322 


L 2,R 1 


0.756 


double scroll 


1.315 


L 1,R 2 


0.761 


double scroll 


1.314 


L 2,R 1 


0.761 



Table V. An example of the transition from spiral to double scroll oscillation pattern of the Chua's 
circuit. 

A2. The entropy produced in the G = 0.7052 system 

The entropy of G = 0.7052, 7 time series simulation data are shown in Table IVTl 
The symbols Sci, Sc2, Sl are the zero mode subtracted Fourier amplitude squared 
of the capacitor 1, capacitor 2 and the inductor. The symbols, e.g. la, lb, Ic 
correspond to the period between the pulses triggered in the sequence 1. 

A3. The entropy produced in the G = 0.7429 system 

The entropy of G = 0.7429, 9 time series simulation data are shown in Table 
IVIII The symbols Sci, Sc2, Sl are the zero mode subtracted Fourier amplitude 
squared of the capacitor 1, capacitor 2 and the inductor. The symbols, e.g. la, lb. 
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sequence 


period of t 


length of the period 


Sci 


Sc2 


Sl 


1 




19.44-38.51 


1908 


1.8617 


0.02805 


2.39917 




la 


19.44-24.19 


476 


0.2067 


0.01916 


0.42803 




lb 


24.20-26.40 


221 


0.39587 


0.05309 


1.50065 




Ic 


26.41-38.51 


1211 


0.2522 


0.02681 


0.57082 


2 




38.52-59.79 


2128 


1.78011 


0.02951 


2.34956 




2a 


38.52-43.34 


483 


0.1635 


0.01399 


0.2815 




2b 


43.35-45.29 


195 


0.22624 


0.04507 


1.12899 




2c 


45.30-59.79 


1450 


0.33889 


0.03246 


0.82155 


3 




59.80-76.57 


1678 


1.65727 


0.02843 


2.19666 




3a 


59.80-62.64 


285 


0.26386 


0.01654 


0.39284 




3b 


62.65-64.60 


196 


0.233 


0.0455 


1.14973 




3c 


64.61-76.57 


1197 


0.2843 


0.02828 


0.63801 


4 




76.58-101.76 


2519 


1.853 


0.02753 


2.37791 




4a 


76.58-79.48 


291 


0.23456 


0.01512 


0.32095 




4b 


79.49-81.33 


185 


0.16841 


0.03863 


0.93344 




4c 


81.34-87.58 


625 


0.56976 


0.02359 


1.02295 




4d 


87.59-89.84 


226 


0.41493 


0.05281 


1.50896 




4e 


89.85-101.76 


1192 


0.2547 


0.02589 


0.55382 


5 




101.77-125.97 


2421 


1.64963 


0.03277 


2.29042 




5a 


101.77-106.52 


476 


0.20028 


0.01854 


0.40951 




5b 


106.53-108.69 


217 


0.37711 


0.05305 


1.48203 




5c 


108.70-125.97 


1728 


0.34053 


0.03405 


0.88238 


6 




125.98-160.12 


3415 


1.8443 


0.02943 


2.41251 




6a 


125.98-128.89 


292 


0.23172 


0.01499 


0.31587 




6b 


128.90-130.72 


183 


0.16209 


0.03752 


0.91628 




6c 


130.73-138.38 


766 


0.42249 


0.01906 


0.79026 




6d 


138.39-140.45 


207 


0.30744 


0.05128 


1.34918 




6e 


140.46-160.12 


1967 


0.35706 


0.03242 


0.88141 


7 




160.13-177.61 


1749 


1.55823 


0.0279 


2.0895 




7a 


160.13-163.46 


334 


0.37432 


0.01719 


0.65062 




7b 


163.47-165.48 


202 


0.27258 


0.04889 


1.26092 




7c 


l()r).i!)-i77.()l 


12i:! 


():2rAr,7 


0.02718 


().r)8i)7:! 



Table VI. The entropy of various oscillation pattern of G = 0.7052 samples 
Ic correspond to the period between the pulses triggered in the sequence 1. 



Entropy of Chaotic Currents in the Chua's Circuit and its HMM Analysis 17 



sequence 


period of t 


length of the period 


Sci 




Sl 


1 




19.25-32.54 


1329 


1.24631 


0.01487 


1.50291 




la 


19.25-22.19 


294 


0.14618 


0.006 


0.16314 




lb 


22.20-24.26 


207 


0.09467 


0.01756 


0.42977 




Ic 


24.27-32.54 


828 


0.02763 


0.01711 


0.4507 


2 




32.55-53.68 


2114 


1.00221 


0.01189 


1.20997 




2a 


32.55-35.68 


309 


0.18756 


0.00814 


0.26958 




2b 


35.69-38.10 


247 


0.21155 


0.02241 


0.65649 




2c 


38.11-53.68 


1558 


0.15711 


0.01091 


0.2747 


3 




53.69-67.16 


1347 


1.22 


0.01578 


1.49536 




3a 


53.69-56.62 


293 


0.1428 


0.00587 


0.15884 




3b 


56.63-58.68 


206 


0.09219 


0.01729 


0.42278 




3c 


58.69-67.16 


848 


0.28869 


0.01861 


0.49672 


4 




67.17-89.08 


2192 


1.0097 


0.00992 


1.17949 




4a 


67.17-70.10 


294 


0.15461 


0.00625 


0.18185 




4b 


70.11-72.25 


215 


0.11608 


0.0193 


0.48424 




4c 


72.26-89.08 


1683 


0.14373 


0.00926 


0.2377 


5 




89.09-105.61 


1653 


1.15233 


0.01411 


1.39947 




5a 


89.09-92.08 


300 


0.17417 


0.00694 


0.23253 




5b 


92.09-94.43 


235 


0.17418 


0.02234 


0.60252 




5c 


94.44-105.61 


1118 


0.19562 


0.01412 


0.34302 


6 




105.62-133.28 


2767 


0.85158 


0.01427 


1.10939 




6a 


105.62-108.66 


305 


0.18279 


0.00724 


0.25467 




6b 


108.67-111.08 


242 


0.19716 


0.02276 


0.63776 




6c 


111.09-133.28 


2220 


0.23936 


0.01424 


0.45321 


7 




133.29-150.71 


1743 


1.06588 


0.01376 


1.30729 




7a 


133.29-137.17 


389 


0.24625 


0.00644 


0.3621 




7b 


137.18-139.52 


235 


0.17805 


0.02235 


0.61051 




7c 


139.53-150.71 


1119 


0.19759 


0.01435 


0.34891 


8 




150.72-167.21 


1650 


1.15656 


0.01408 


1.40273 




8a 


150.72-153.70 


299 


0.17213 


0.00687 


0.22618 




8b 


153.70-156.02 


232 


0.16718 


0.022 


0.59081 




8c 


156.02-167.21 


1119 


0.19775 


0.0142 


0.34696 


9 




167.22-197.70 


3049 


0.80684 


0.01508 


1.07902 




9a 


167.22-170.25 


304 


0.18018 


0.0072 


0.24926 




9b 


170.26-172.65 


240 


0.1925 


0.02269 


0.63215 




9c 


172.66-197.70 


2505 


0.24864 


0.01524 


0.48473 



Table VII. The entropy of various oscillation patterns of G = 0.7429 samples. 



References 

S. Furui and S. Niiya , Shil'nikov Chaos control using Hornoclinic orbits and the Ncwhousc 
region,@Chaos, Sohtons & Fractals , 34(2007), 966, arXiv:nlinCD/0604014. 
L.O. Chua , M. Komuro and M. Matsumoto , The double scroll family, IEEE Trans. Circ. 
Syst. ,CAS ,33(1986), 1073. 

R.P. Feynman , R.B. Leighton , and M. Sands , The Feynman Lectures on Physics, 
Addison- Wesley: vol II p.19-14 (1965) 

L.D. Landau and E.M. Lifshitz C Statistical P/iysicsC (Translated into Japanese by 

Kobayashi ct al.)CIwanami-shotcn (1966). 

I. Prigoginc and D. Kondopty , Modern Thermodynamics, (Translated into Japanese by 
Senoo and Iwamoto) Asakura-shoten, (2001). 

J.J. Kay , About some slipups in applying Prigogine's minimum entropy production prin- 



1) 
2) 
3) 
4) 
5) 
6) 



Sadataka Furui 



ciple to living systems, 'htt p : / / www . j ameskay. ca/ musings /mep . p df " 

R. Landauer , Inadequacy of entropy and entropy derivatives in characterizing the staedy 
state, Phys. Rev. A12(1975),636 . 

S. Bruers , Ch. Maes and K. Netocny , On the varidity of entropy production principle for 
linear electrical circuits, arXiv:cond- mat/0701035 

R. Dewar, Maximum entropy production and the fluctuation theorem, J. Phys. A38(2005), 
L371-L381 . 

S. Bruers , A discussion on maximum entropy production and information theory, J. Stat. 
Phys. 129(2007), 725, arXiv:0705.3226 cond-mat.stat-mech]. 

Ch. Maes and K. Netocny , Minimum entropy production principle froma dynamical fluc- 
tuation law, J. Math. Phys. 48 (2007), 053306, arXiv:math-ph/0612063 
R. Bowen, Equilibrium States and the Ergodic Theory of Anosov Diffeomorphisms, Lecture 
Notes in Mathematics, Ed. by Chazottes J-R., (2nd Revised Edition) Springer (2008). 
S.E. Newhouse, The abundance of wild hyperbolic sets and non-smooth stable sets for 
diffeomorphisms, Publ. Matth. IHES 50 (1979), 101. 

J. P. Sethna, Statistical Mechanics: Entropy, Order Parameters, and Complexity, Oxford 
Univ. Press, (2006). 

C. Robinson, Dynamical Systems: Stability, Symbolic Dynamics, and Chaos, CRC Press, 
(1998), Translation into Japanese by Kokubu et al.. Springer Verlag, (2001). 
M. Boyle, Notes on the Perron-Frobenius Theory of Nonnegative Matrices, 
http: / /www. math. umb.edu/~mmb 

L.E. Baum , T. Petrie , G. Soules and N. Weiss , A Maximization Technique Occuring 
in the Statistical Analysis of Probabilistic Function of Markov Chains, Ann. Math. Stat. 
41(1970), 164 . 

L.R. Welch, Hidden Markov Models and the Baum- Welch Algorithm, IEEE Information 
Theory Society Newsletter, 2003; vol.53, 9 . 

S. Hayes , C. Grebogi and E. Ott , Communicating with Chaos, Phys. Rev. 
Lett. 70(1993), 3031. 

A. Dimitriadis and A.M. Eraser, HEEE Transactions on circuits and Systems-II: Analog 
and Digital Signal Processing, vol.40 (1993), 683. 

S. Furui, The Color Antiymmetric Ghost Propagator and One-Loop Vertex Renormaliza- 
tion, Prog. Theor. Phys.(Kyoto)119(2008),149, arXiv:0709.2804;hep-ph]. 
N. Hoshi, About Chaos genarated by Electric Circuit, Master Thesis, Graduate School of 
Teikyo University, (2010). 



